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Abstract 

A set of fixed points of the Hopfield type neural network was under investigation. 
Its connection matrix is constructed with regard to the Hebb rule from a highly 
symmetric set of the memorized patterns. Depending on the external parameter the 
analytic description of the fixed points set had been obtained. And as a conclusion, 
some exact results of Hopfield neural networks were gained. 
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Over the past decade, there has been an explosion of interest in so-called artificial neural 
network(ANN) technology. ANNs are a model of computing inspired by the brain[l]-[5], 
consisting of a collection of model "neurons" connected by model "synapses." Compu- 
tation in the network is performed in a distributed fashion, by propagating excitatory 
and inhibitory activations across the synapses, and computing the neuronal outputs as a 
nonlinear (typically sigmoidal) function of total synaptic input. These networks also have 
a capacity to "learn" to perform a given computation by adjusting real- valued synaptic 
connection strengths (weight values) between units. ANNs are of considerable interest 
both for biological modeling of information processing in the nervous system, and for 

solving many classes of complex real-world applications. 
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J.J.Hopfield boosted neural network research at the beginning of the 1980s with the 
publication of a famous paper on artificial neural networks, which he used for pattern 
completion and to solve optimization problems [4]. These networks consist of one layer of 
neurons that are completely connected with each other. Hopfield analysed the behavior 
of networks belonging to that type, and could prove mathematically that stable behavior 
may be achieved under certain conditions. 

It can be shown that the dynamic behavior of Hopfield type neural networks is de- 
scribed by an energe surface. Each network state corresponds to a certain position on 
that surface. Through external clamping, neurons may be forced to certain states of 
activity, and thus the whole network may be forced to move to a well defined point on 
the energy surface. If the network is released, i.e. external clamping is removed, it will 
change its state in such a way that it moves on the energy surface towards new states of 
lower energy. Finally, neuron states will stop changing if a local minimum in the energy 
surface is reached. Through careful selection of weights, ocillations will be avoided. A 
set of fixed points of the Hopfield type neural network [4] [6] is under investigation. Its 
connection matrix is constructed with regard to the Hebb rule from a (p x n) -matrix S 
of memorized patterns: 



S = 



/ 1-x 1 ... 1 1 ... 1 \ 
1 1-x ... 1 1 ... 1 



V 1 1 ... 1 -x 1 ... 1 / 

Here n is the number of neurons, p is the number of memorized patterns s^ l \ which are 
the rows of the matrix S, and x is an arbitrary real number. 

Depending on x the memorized patterns are interpreted as p distorted vectors of 
the standard 

£» = (1,1,...,1). (1) 

n 

We denote by e(k) the configuration vector which is collinear to the bisectrix of the 
principle orthant standard-vector. Next, n is the number of the spin variables, p is the 
number of the memorized patterns and q = n — p is the number of the nondistorted 
coordinates of the standard-vector. Configuration vectors are denoted by small Greek 
letters. We use small Latin letters to denote vectors whose coordinates are real. 
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The problem is as follows: the network has to be learned by p-times showing of the 
standard (1), but a distortion has slipped in the learning process. How does the fixed points 
set depends on the value of this distortion x? 

Depending on the distortion parameter x the analytic description of the fixed points 
set has been obtained. It turns out to be very important that the memorized patterns 
§(0 form a highly symmetric group of vectors: all of them correlate one with another in 
the same way: 

(s<') j5 (i')) = r (x), (2) 

where r(x) is independent of 1,1' = 1,2,..., p. Namely this was the reason to use the 
words "highly symmetric" in the title. 

It is known [7] , that the fixed points of a network of our kind have to be of the form: 

a* = ((7i,(72,..., <7 p ,l,...,l), <Ti = {±1}, % = 1,2,..., p. (3) 

Let's join into one class £^ all the configuration vectors a* given by Eq.(3), which have 
k coordinates equal to among the first p coordinates. The class consists of Cp 
configuration vectors of the form (3), and there are p+1 different classes (k — 0, 1, . . . ,p). 
Our main result can be formulated as a Theorem. 

Theorem. As x varies from — oo to oo the fixed points set is exhausted in consecutive 
order by the classes of the vectors[8] 

and the transformation of the fixed points set from the class E^ -1 ) into the class T.^ 
occurs when x = x^: 

n-(2k-l) , , n 

X ^=P — , 7v k=l,2,...,K. 

n + p — 2(2k — 1) 

If ^zi < \> according this scheme all the p transformations of the fixed points set are 
realized one after another and K = p. If > |> the transformation related to 

~n + p + 2" 



K 



4 

is the last. The network has no other fixed points. 
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The Theorem makes it possible to solve a number of practical problems. We would 
like to add that the Theorem can be generalized onto the case of arbitrary vector 

v 

U = (ui,U2,...,Up,l,...,l), J2 u2 i=P 

i=l 

being a standard instead the standard (1). Here memorized patterns are obtained by 
the distortion of the first p coordinates of the vector u with regard to the fulfillment of 
Eqs.(2). 

The obtained results can be interpreted in terms of neural networks, Ising model and 
factor analysis. 
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